Three-dimensional point-in-polygon operation to facilitate displaying three-dimensional structures

ABSTRACT

A system, a method and instructions embodied on a non-transitory computer-readable storage medium that solve a 3D point-in-polygon (PIP) problem is presented. This system projects polygons that comprise a set of polyhedra onto projected polygons in a reference plane. Next, the system projects a data point onto the reference plane, and performs a 2D PIP operation in the reference plane to determine which projected polygons the projected data point falls into. For each projected polygon the projected data point falls into, the system performs a 3D crossing number operation by counting intersections between a ray projected from the corresponding data point in a direction orthogonal to the reference plane and polyhedral faces corresponding to projected polygons, to identify polyhedra the data point falls into. The system then generates a visual representation of the set of polyhedra, wherein each polyhedron is affected by data points that fall into it.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/700,685, filed Apr. 30, 2015, entitled “Three DimensionalPoint-In-Polygon Operation To Facilitate Displaying Three-DimensionalStructures,” which is a continuation-in-part of, and hereby claimspriority to U.S. application Ser. No. 14/606,387, filed Jan. 27, 2015,entitled “Efficient Point-In-Polygon Indexing Technique to FacilitateDisplaying Geographic Data,” which is incorporated in its entirety byreference herein.

BACKGROUND OF THE INVENTION

Field of the Invention

The disclosed embodiments generally relate to techniques for displayingthree-dimensional data sets. More specifically, the disclosedembodiments relate to a system that efficiently solves athree-dimensional point in-polygon problem to facilitate displayingthree-dimensional structures.

Related Art

Solving the point-in-polygon (PIP) problem involves determining whethera point in a two-dimensional space falls within one or more polygons inthe two-dimensional space. Solutions to the PIP problem have importantapplications in a variety of disciplines, including geographicinformation systems (GISs), computer games, computer vision applicationsand computer-aided design (CAD). A number of systems have presently beendeveloped to efficiently solve the two-dimensional PIP problem.

An interesting variation of the two-dimensional PIP problem is thethree-dimensional PIP problem that seeks to determine whether a point ina three dimensional space falls within one or more polyhedra in thethree-dimensional space. This three-dimensional PIP problem hasimportant applications in a variety of disciplines, including computinggaming and medical imaging. However, no systems have been developed toefficiently solve the three-dimensional PIP problem.

Hence, what is needed is a system that efficiently solves the threedimensional PIP problem.

BRIEF SUMMARY OF THE INVENTION

The disclosed embodiments relate to a system, a method and instructionsembodied on a non-transitory computer-readable storage medium that solvea 3D point-in-polygon (PIP) problem is presented. This system projectspolygons that comprise a set of polyhedra onto projected polygons in areference plane. Then, for each data point to be processed, the systemprojects the data point onto the reference plane, and performs a 2D PIPoperation in the reference plane to determine which projected polygonsthe projected data point falls into. For each projected polygon that theprojected data point falls into, the system performs a 3D crossingnumber operation by counting intersections between a ray projected fromthe corresponding data point in a direction orthogonal to the referenceplane and polyhedral faces corresponding to projected polygons, toidentify zero or more polyhedra the data point falls into. The systemthen generates a visual representation of the set of polyhedra, whereineach polyhedron is affected by data points that fall into thepolyhedron.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIG. 1 presents a block diagram of an event-processing system inaccordance with the disclosed embodiments.

FIG. 2 presents a flowchart illustrating how indexers process, index,and store data received from forwarders in accordance with the disclosedembodiments.

FIG. 3 presents a flowchart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments.

FIG. 4 presents a block diagram of a system for processing searchrequests that uses extraction rules for field values in accordance withthe disclosed embodiments.

FIG. 5 illustrates an exemplary search query received from a client andexecuted by search peers in accordance with the disclosed embodiments.

FIG. 6A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 6B illustrates a data summary dialog that enables a user to selectvarious data sources in accordance with the disclosed embodiments.

FIG. 7 illustrates a system for processing and displaying geographicdata in accordance with the disclosed embodiments.

FIG. 8 illustrates a ray being cast across a polygon with crossingnumber (CN) counts in accordance with the disclosed embodiments.

FIG. 9 illustrates polygon segments projected onto a Y-axis inaccordance with the disclosed embodiments.

FIG. 10 illustrates horizontal rays cast in each Y-range in accordancewith the disclosed embodiments.

FIG. 11 illustrates segment projections with multiple polygons inaccordance with the disclosed embodiments.

FIG. 12 illustrates a grouping of intersection sets by polygon identityin accordance with the disclosed embodiments.

FIG. 13 presents a flow chart illustrating the process of displayinggeographic data in accordance with the disclosed embodiments.

FIG. 14 presents a flow chart illustrating the process of performing ageofencing operation in accordance with the disclosed embodiments.

FIG. 15 presents a flow chart illustrating the processing of a queryinvolving geographic data in accordance with the disclosed embodiments.

FIG. 16 illustrates an exemplary instance of a three-dimensional PIPproblem in accordance with the disclosed embodiments.

FIG. 17 presents a flow chart illustrating the process of solving athree-dimensional PIP problem in accordance with the disclosedembodiments.

Table 1 illustrates sorted point structures in accordance with thedisclosed embodiments.

Table 2 illustrates sorted point structures with corresponding lists ofopen segments in accordance with the disclosed embodiments.

Table 3 illustrates how non-final rows can be struck out in accordancewith the disclosed embodiments.

Table 4 illustrates Y-ranges and corresponding open segments inaccordance with the disclosed embodiments.

Table 5 illustrates how non-final rows can be struck out in accordancewith the disclosed embodiments.

Table 6 illustrates how the data structure can be modified for modulo 3storage in accordance with the disclosed embodiments.

DETAILED DESCRIPTION OF THE INVENTION

The disclosed embodiments relate to a system, a method and instructionsembodied on a non-transitory computer-readable storage medium code thatfacilitate displaying three-dimensional structures. (Note thatthroughout this specification and the attached claims we refer to thesystem, the method and the instructions embodied on a non-transitorycomputer-readable storage medium collectively as “the system.”) Thissystem is described in more detail below, but we first describe thestructure of an event-based framework in which this system operates.

1.1 System Overview

Modern data centers often comprise thousands of host computer systemsthat operate collectively to service requests from even larger numbersof remote clients. During operation, these data centers generatesignificant volumes of performance data and diagnostic information thatcan be analyzed to quickly diagnose performance problems. In order toreduce the size of this performance data, the data is typicallypre-processed prior to being stored based on anticipated data-analysisneeds. For example, pre-specified data items can be extracted from theperformance data and stored in a database to facilitate efficientretrieval and analysis at search time. However, the rest of theperformance data is not saved and is essentially discarded duringpre-processing. As storage capacity becomes progressively cheaper andmore plentiful, there are fewer incentives to discard this performancedata and many reasons to keep it.

This plentiful storage capacity is presently making it feasible to storemassive quantities of minimally processed performance data at “ingestiontime” for later retrieval and analysis at “search time.” Note thatperforming the analysis operations at search time provides greaterflexibility because it enables an analyst to search all of theperformance data, instead of searching pre-specified data items thatwere stored at ingestion time. This enables the analyst to investigatedifferent aspects of the performance data instead of being confined tothe pre-specified set of data items that was selected at ingestion time.

However, analyzing massive quantities of heterogeneous performance dataat search time can be a challenging task. A data center may generateheterogeneous performance data from thousands of different components,which can collectively generate tremendous volumes of performance datathat can be time-consuming to analyze. For example, this performancedata can include data from system logs, network packet data, sensordata, and data generated by various applications. Also, the unstructurednature of much of this performance data can pose additional challengesbecause of the difficulty of applying semantic meaning to unstructureddata, and the difficulty of indexing and querying unstructured datausing traditional database systems.

These challenges can be addressed by using an event-based system, suchas the SPLUNK® ENTERPRISE system produced by Splunk Inc. of SanFrancisco, Calif., to store and process performance data. The SPLUNK®ENTERPRISE system is the leading platform for providing real-timeoperational intelligence that enables organizations to collect, index,and harness machine-generated data from various websites, applications,servers, networks, and mobile devices that power their businesses. TheSPLUNK® ENTERPRISE system is particularly useful for analyzingunstructured performance data, which is commonly found in system logfiles. Although many of the techniques described herein are explainedwith reference to the SPLUNK® ENTERPRISE system, the techniques are alsoapplicable to other types of data server systems.

In the SPLUNK® ENTERPRISE system, performance data is stored as“events,” wherein each event comprises a collection of performance dataand/or diagnostic information that is generated by a computer system andis correlated with a specific point in time. Events can be derived from“time series data,” wherein time series data comprises a sequence ofdata points (e.g., performance measurements from a computer system) thatare associated with successive points in time and are typically spacedat uniform time intervals. Events can also be derived from “structured”or “unstructured” data. Structured data has a predefined format, whereinspecific data items with specific data formats reside at predefinedlocations in the data. For example, structured data can include dataitems stored in fields in a database table. In contrast, unstructureddata does not have a predefined format. This means that unstructureddata can comprise various data items having different data types thatcan reside at different locations. For example, when the data source isan operating system log, an event can include one or more lines from theoperating system log containing raw data that includes different typesof performance and diagnostic information associated with a specificpoint in time. Examples of data sources from which an event may bederived include, but are not limited to: web servers; applicationservers; databases; firewalls; routers; operating systems; and softwareapplications that execute on computer systems, mobile devices, andsensors. The data generated by such data sources can be produced invarious forms including, for example and without limitation, server logfiles, activity log files, configuration files, messages, network packetdata, performance measurements and sensor measurements. An eventtypically includes a timestamp that may be derived from the raw data inthe event, or may be determined through interpolation between temporallyproximate events having known timestamps.

The SPLUNK® ENTERPRISE system also facilitates using a flexible schemato specify how to extract information from the event data, wherein theflexible schema may be developed and redefined as needed. Note that aflexible schema may be applied to event data “on the fly,” when it isneeded (e.g., at search time), rather than at ingestion time of the dataas in traditional database systems. Because the schema is not applied toevent data until it is needed (e.g., at search time), it is referred toas a “late-binding schema.”

During operation, the SPLUNK® ENTERPRISE system starts with raw data,which can include unstructured data, machine data, performancemeasurements or other time-series data, such as data obtained fromweblogs, syslogs, or sensor readings. It divides this raw data into“portions,” and optionally transforms the data to produce time stampedevents. The system stores the time stamped events in a data store, andenables a user to run queries against the data store to retrieve eventsthat meet specified criteria, such as containing certain keywords orhaving specific values in defined fields. Note that the term “field”refers to a location in the event data containing a value for a specificdata item.

As noted above, the SPLUNK® ENTERPRISE system facilitates using alate-binding schema while performing queries on events. A late-bindingschema specifies “extraction rules” that are applied to data in theevents to extract values for specific fields. More specifically, theextraction rules for a field can include one or more instructions thatspecify how to extract a value for the field from the event data. Anextraction rule can generally include any type of instruction forextracting values from data in events. In some cases, an extraction rulecomprises a regular expression, in which case the rule is referred to asa “regex rule.”

In contrast to a conventional schema for a database system, alate-binding schema is not defined at data ingestion time. Instead, thelate-binding schema can be developed on an ongoing basis until the timea query is actually executed. This means that extraction rules for thefields in a query may be provided in the query itself, or may be locatedduring execution of the query. Hence, as an analyst learns more aboutthe data in the events, the analyst can continue to refine thelate-binding schema by adding new fields, deleting fields, or changingthe field extraction rules until the next time the schema is used by aquery. Because the SPLUNK® ENTERPRISE system maintains the underlyingraw data and provides a late-binding schema for searching the raw data,it enables an analyst to investigate questions that arise as the analystlearns more about the events.

In the SPLUNK® ENTERPRISE system, a field extractor may be configured toautomatically generate extraction rules for certain fields in the eventswhen the events are being created, indexed, or stored, or possibly at alater time. Alternatively, a user may manually define extraction rulesfor fields using a variety of techniques.

Also, a number of “default fields” that specify metadata about theevents rather than data in the events themselves can be createdautomatically. For example, such default fields can specify: a timestampfor the event data; a host from which the event data originated; asource of the event data; and a source type for the event data. Thesedefault fields may be determined automatically when the events arecreated, indexed or stored.

In some embodiments, a common field name may be used to reference two ormore fields containing equivalent data items, even though the fields maybe associated with different types of events that possibly havedifferent data formats and different extraction rules. By enabling acommon field name to be used to identify equivalent fields fromdifferent types of events generated by different data sources, thesystem facilitates use of a “common information model” (CIM) across thedifferent data sources.

1.2 Data Server System

FIG. 1 presents a block diagram of an exemplary event-processing system100, similar to the SPLUNK® ENTERPRISE system. System 100 includes oneor more forwarders 101 that collect data obtained from a variety ofdifferent data sources 105, and one or more indexers 102 that store,process, and/or perform operations on this data, wherein each indexeroperates on data contained in a specific data store 103. Theseforwarders and indexers can comprise separate computer systems in a datacenter, or may alternatively comprise separate processes executing onvarious computer systems in a data center.

During operation, the forwarders 101 identify which indexers 102 willreceive the collected data and then forward the data to the identifiedindexers. Forwarders 101 can also perform operations to strip outextraneous data and detect timestamps in the data. The forwarders nextdetermine which indexers 102 will receive each data item and thenforward the data items to the determined indexers 102.

Note that distributing data across different indexers facilitatesparallel processing. This parallel processing can take place at dataingestion time, because multiple indexers can process the incoming datain parallel. The parallel processing can also take place at search time,because multiple indexers can search through the data in parallel.

System 100 and the processes described below with respect to FIGS. 1-5are further described in “Exploring Splunk Search Processing Language(SPL) Primer and Cookbook” by David Carasso, CITO Research, 2012, and in“Optimizing Data Analysis With a Semi-Structured Time Series Database”by Ledion Bitincka, Archana Ganapathi, Stephen Sorkin, and Steve Zhang,SLAML, 2010, each of which is hereby incorporated herein by reference inits entirety for all purposes.

1.3 Data Ingestion

FIG. 2 presents a flow chart illustrating how an indexer processes,indexes, and stores data received from forwarders in accordance with thedisclosed embodiments. At block 201, the indexer receives the data fromthe forwarder. Next, at block 202, the indexer apportions the data intoevents. Note that the data can include lines of text that are separatedby carriage returns or line breaks and an event may include one or moreof these lines. During the apportioning process, the indexer can useheuristic rules to automatically determine the boundaries of the events,which for example coincide with line boundaries. These heuristic rulesmay be determined based on the source of the data, wherein the indexercan be explicitly informed about the source of the data or can infer thesource of the data by examining the data. These heuristic rules caninclude regular expression-based rules or delimiter-based rules fordetermining event boundaries, wherein the event boundaries may beindicated by predefined characters or character strings. Thesepredefined characters may include punctuation marks or other specialcharacters including, for example, carriage returns, tabs, spaces orline breaks. In some cases, a user can fine-tune or configure the rulesthat the indexers use to determine event boundaries in order to adaptthe rules to the user's specific requirements.

Next, the indexer determines a timestamp for each event at block 203. Asmentioned above, these timestamps can be determined by extracting thetime directly from data in the event, or by interpolating the time basedon timestamps from temporally proximate events. In some cases, atimestamp can be determined based on the time the data was received orgenerated. The indexer subsequently associates the determined timestampwith each event at block 204, for example by storing the timestamp asmetadata for each event.

Then, the system can apply transformations to data to be included inevents at block 205. For log data, such transformations can includeremoving a portion of an event (e.g., a portion used to define eventboundaries, extraneous text, characters, etc.) or removing redundantportions of an event. Note that a user can specify portions to beremoved using a regular expression or any other possible technique.

Next, a keyword index can optionally be generated to facilitate fastkeyword searching for events. To build a keyword index, the indexerfirst identifies a set of keywords in block 206. Then, at block 207 theindexer includes the identified keywords in an index, which associateseach stored keyword with references to events containing that keyword(or to locations within events where that keyword is located). When anindexer subsequently receives a keyword based query, the indexer canaccess the keyword index to quickly identify events containing thekeyword.

In some embodiments, the keyword index may include entries forname-value pairs found in events, wherein a name-value pair can includea pair of keywords connected by a symbol, such as an equals sign orcolon. In this way, events containing these name-value pairs can bequickly located. In some embodiments, fields can automatically begenerated for some or all of the name value pairs at the time ofindexing. For example, if the string “dest=10.0.1.2” is found in anevent, a field named “dest” may be created for the event, and assigned avalue of “10.0.1.2.”

Finally, the indexer stores the events in a data store at block 208,wherein a timestamp can be stored with each event to facilitatesearching for events based on a time range. In some cases, the storedevents are organized into a plurality of buckets, wherein each bucketstores events associated with a specific time range. This not onlyimproves time-based searches, but it also allows events with recenttimestamps that may have a higher likelihood of being accessed to bestored in faster memory to facilitate faster retrieval. For example, abucket containing the most recent events can be stored as flash memoryinstead of on hard disk.

Each indexer 102 is responsible for storing and searching a subset ofthe events contained in a corresponding data store 103. By distributingevents among the indexers and data stores, the indexers can analyzeevents for a query in parallel, for example using map-reduce techniques,wherein each indexer returns partial responses for a subset of events toa search head that combines the results to produce an answer for thequery. By storing events in buckets for specific time ranges, an indexermay further optimize searching by looking only in buckets for timeranges that are relevant to a query.

Moreover, events and buckets can also be replicated across differentindexers and data stores to facilitate high availability and disasterrecovery as is described in U.S. patent application Ser. No. 14/266,812filed on 30 Apr. 2014, and in U.S. patent application Ser. No.14/266,817 also filed on 30 Apr. 2014.

1.4 Query Processing

FIG. 3 presents a flow chart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments. Atthe start of this process, a search head receives a search query from aclient at block 301. Next, at block 302, the search head analyzes thesearch query to determine what portions can be delegated to indexers andwhat portions need to be executed locally by the search head. At block303, the search head distributes the determined portions of the query tothe indexers. Note that commands that operate on single events can betrivially delegated to the indexers, while commands that involve eventsfrom multiple indexers are harder to delegate.

Then, at block 304, the indexers to which the query was distributedsearch their data stores for events that are responsive to the query. Todetermine which events are responsive to the query, the indexer searchesfor events that match the criteria specified in the query. This criteriacan include matching keywords or specific values for certain fields. Ina query that uses a late-binding schema, the searching operations inblock 304 may involve using the late-binding scheme to extract valuesfor specified fields from events at the time the query is processed.Next, the indexers can either send the relevant events back to thesearch head, or use the events to calculate a partial result, and sendthe partial result back to the search head.

Finally, at block 305, the search head combines the partial resultsand/or events received from the indexers to produce a final result forthe query. This final result can comprise different types of datadepending upon what the query is asking for. For example, the finalresults can include a listing of matching events returned by the query,or some type of visualization of data from the returned events. Inanother example, the final result can include one or more calculatedvalues derived from the matching events.

Moreover, the results generated by system 100 can be returned to aclient using different techniques. For example, one technique streamsresults back to a client in real-time as they are identified. Anothertechnique waits to report results to the client until a complete set ofresults is ready to return to the client. Yet another technique streamsinterim results back to the client in real-time until a complete set ofresults is ready, and then returns the complete set of results to theclient. In another technique, certain results are stored as “searchjobs,” and the client may subsequently retrieve the results byreferencing the search jobs.

The search head can also perform various operations to make the searchmore efficient. For example, before the search head starts executing aquery, the search head can determine a time range for the query and aset of common keywords that all matching events must include. Next, thesearch head can use these parameters to query the indexers to obtain asuperset of the eventual results. Then, during a filtering stage, thesearch head can perform field extraction operations on the superset toproduce a reduced set of search results.

1.5 Field Extraction

FIG. 4 presents a block diagram illustrating how fields can be extractedduring query processing in accordance with the disclosed embodiments. Atthe start of this process, a search query 402 is received at a queryprocessor 404. Query processor 404 includes various mechanisms forprocessing a query, wherein these mechanisms can reside in a search head104 and/or an indexer 102. Note that the exemplary search query 402illustrated in FIG. 4 is expressed in the Search Processing Language(SPL), which is used in conjunction with the SPLUNK® ENTERPRISE system.SPL is a pipelined search language in which a set of inputs is operatedon by a first command in a command line, and then a subsequent commandfollowing the pipe symbol “I” operates on the results produced by thefirst command, and so on for additional commands. Search query 402 canalso be expressed in other query languages, such as the Structured QueryLanguage (“SQL”) or any suitable query language.

Upon receiving search query 402, query processor 404 sees that searchquery 402 includes two fields “IP” and “target.” Query processor 404also determines that the values for the “IP” and “target” fields havenot already been extracted from events in data store 414, andconsequently determines that query processor 404 needs to use extractionrules to extract values for the fields. Hence, query processor 404performs a lookup for the extraction rules in a rule base 406, whereinrule base 406 maps field names to corresponding extraction rules andobtains extraction rules 408-409, wherein extraction rule 408 specifieshow to extract a value for the “IP” field from an event, and extractionrule 409 specifies how to extract a value for the “target” field from anevent. As is illustrated in FIG. 4, extraction rules 408-409 cancomprise regular expressions that specify how to extract values for therelevant fields. Such regular-expression-based extraction rules are alsoreferred to as “regex rules.” In addition to specifying how to extractfield values, the extraction rules may also include instructions forderiving a field value by performing a function on a character string orvalue retrieved by the extraction rule. For example, a transformationrule may truncate a character string, or convert the character stringinto a different data format. In some cases, the query itself canspecify one or more extraction rules.

Next, query processor 404 sends extraction rules 408-409 to a fieldextractor 412, which applies extraction rules 408-409 to events 416-418in a data store 414. Note that data store 414 can include one or moredata stores, and extraction rules 408-409 can be applied to largenumbers of events in data store 414, and are not meant to be limited tothe three events 416-418 illustrated in FIG. 4. Moreover, the queryprocessor 404 can instruct field extractor 412 to apply the extractionrules to all the events in a data store 414, or to a subset of theevents that has been filtered based on some criteria.

Next, field extractor 412 applies extraction rule 408 for the firstcommand Search IP=“10*” to events in data store 414 including events416-418. Extraction rule 408 is used to extract values for the IPaddress field from events in data store 414 by looking for a pattern ofone or more digits, followed by a period, followed again by one or moredigits, followed by another period, followed again by one or moredigits, followed by another period, and followed again by one or moredigits. Next, field extractor 412 returns field values 420 to queryprocessor 404, which uses the criterion IP=“10*” to look for IPaddresses that start with “10”. Note that events 416 and 417 match thiscriterion, but event 418 does not, so the result set for the firstcommand is events 416-417.

Query processor 404 then sends events 416-417 to the next command “statscount target.” To process this command, query processor 404 causes fieldextractor 412 to apply extraction rule 409 to events 416-417. Extractionrule 409 is used to extract values for the target field for events416-417 by skipping the first four commas in events 416-417, and thenextracting all of the following characters until a comma or period isreached. Next, field extractor 412 returns field values 421 to queryprocessor 404, which executes the command “stats count target” to countthe number of unique values contained in the target fields, which inthis example produces the value “2” that is returned as a final result422 for the query.

Note that query results can be returned to a client, a search head, orany other system component for further processing. In general, queryresults may include: a set of one or more events; a set of one or morevalues obtained from the events; a subset of the values; statisticscalculated based on the values; a report containing the values; or avisualization, such as a graph or chart, generated from the values.

1.6 Exemplary Search Screen

FIG. 6A illustrates an exemplary search screen 600 in accordance withthe disclosed embodiments. Search screen 600 includes a search bar 602that accepts user input in the form of a search string. It also includesa time range picker 612 that enables the user to specify a time rangefor the search. For “historical searches” the user can select a specifictime range, or alternatively a relative time range, such as “today,”“yesterday” or “last week.” For “real-time searches,” the user canselect the size of a preceding time window to search for real-timeevents. Search screen 600 also initially displays a “data summary”dialog as is illustrated in FIG. 6B that enables the user to selectdifferent sources for the event data, for example by selecting specifichosts and log files.

After the search is executed, the search screen 600 can display theresults through search results tabs 604, wherein search results tabs 604includes: an “events tab” that displays various information about eventsreturned by the search; a “statistics tab” that displays statisticsabout the search results; and a “visualization tab” that displaysvarious visualizations of the search results. The events tab illustratedin FIG. 6A displays a timeline graph 605 that graphically illustratesthe number of events that occurred in one-hour intervals over theselected time range. It also displays an events list 608 that enables auser to view the raw data in each of the returned events. Itadditionally displays a fields sidebar 606 that includes statisticsabout occurrences of specific fields in the returned events, including“selected fields” that are pre-selected by the user, and “interestingfields” that are automatically selected by the system based onpre-specified criteria.

1.7 Acceleration Techniques

The above-described system provides significant flexibility by enablinga user to analyze massive quantities of minimally processed performancedata “on the fly” at search time instead of storing pre-specifiedportions of the performance data in a database at ingestion time. Thisflexibility enables a user to see correlations in the performance dataand perform subsequent queries to examine interesting aspects of theperformance data that may not have been apparent at ingestion time.

However, performing extraction and analysis operations at search timecan involve a large amount of data and require a large number ofcomputational operations, which can cause considerable delays whileprocessing the queries. Fortunately, a number of acceleration techniqueshave been developed to speed up analysis operations performed at searchtime. These techniques include: (1) performing search operations inparallel by formulating a search as a map-reduce computation; (2) usinga keyword index; (3) using a high performance analytics store; and (4)accelerating the process of generating reports. These techniques aredescribed in more detail below.

1.7.1 Map-Reduce Technique

To facilitate faster query processing, a query can be structured as amap-reduce computation, wherein the “map” operations are delegated tothe indexers, while the corresponding “reduce” operations are performedlocally at the search head. For example, FIG. 5 illustrates how a searchquery 501 received from a client at search head 104 can split into twophases, including: (1) a “map phase” comprising subtasks 502 (e.g., dataretrieval or simple filtering) that may be performed in parallel and are“mapped” to indexers 102 for execution, and (2) a “reduce phase”comprising a merging operation 503 to be executed by the search headwhen the results are ultimately collected from the indexers.

During operation, upon receiving search query 501, search head 104modifies search query 501 by substituting “stats” with “prestats” toproduce search query 502, and then distributes search query 502 to oneor more distributed indexers, which are also referred to as “searchpeers.” Note that search queries may generally specify search criteriaor operations to be performed on events that meet the search criteria.Search queries may also specify field names, as well as search criteriafor the values in the fields or operations to be performed on the valuesin the fields. Moreover, the search head may distribute the full searchquery to the search peers as is illustrated in FIG. 3, or mayalternatively distribute a modified version (e.g., a more restrictedversion) of the search query to the search peers. In this example, theindexers are responsible for producing the results and sending them tothe search head. After the indexers return the results to the searchhead, the search head performs the merging operations 503 on theresults. Note that by executing the computation in this way, the systemeffectively distributes the computational operations while minimizingdata transfers.

1.7.2 Keyword Index

As described above with reference to the flow charts in FIGS. 2 and 3,event-processing system 100 can construct and maintain one or morekeyword indices to facilitate rapidly identifying events containingspecific keywords. This 25 can greatly speed up the processing ofqueries involving specific keywords. As mentioned above, to build akeyword index, an indexer first identifies a set of keywords. Then, theindexer includes the identified keywords in an index, which associateseach stored keyword with references to events containing that keyword,or to locations within events where that keyword is located. When anindexer subsequently receives a keyword-based query, the indexer canaccess the keyword index to quickly identify events containing thekeyword.

1.7.3 High Performance Analytics Store

To speed up certain types of queries, some embodiments of system 100make use of a high performance analytics store, which is referred to asa “summarization table,” that contains entries for specific field-valuepairs. Each of these entries keeps track of instances of a specificvalue in a specific field in the event data and includes references toevents containing the specific value in the specific field. For example,an exemplary entry in a summarization table can keep track ofoccurrences of the value “94107” in a “ZIP code” field of a set ofevents, wherein the entry includes references to all of the events thatcontain the value “94107” in the ZIP code field. This enables the systemto quickly process queries that seek to determine how many events have aparticular value for a particular field, because the system can examinethe entry in the summarization table to count instances of the specificvalue in the field without having to go through the individual events ordo extractions at search time. Also, if the system needs to process allevents that have a specific field-value combination, the system can usethe references in the summarization table entry to directly access theevents to extract further information without having to search all ofthe events to find the specific field-value combination at search time.

In some embodiments, the system maintains a separate summarization tablefor each of the above-described time-specific buckets that stores eventsfor a specific time range, wherein a bucket-specific summarization tableincludes entries for specific field-value combinations that occur inevents in the specific bucket. Alternatively, the system can maintain aseparate summarization table for each indexer, wherein theindexer-specific summarization table only includes entries for theevents in a data store that is managed by the specific indexer.

The summarization table can be populated by running a “collection query”that scans a set of events to find instances of a specific field-valuecombination, or alternatively instances of all field-value combinationsfor a specific field. A collection query can be initiated by a user, orcan be scheduled to occur automatically at specific time intervals. Acollection query can also be automatically launched in response to aquery that asks for a specific field-value combination.

In some cases, the summarization tables may not cover all of the eventsthat are relevant to a query. In this case, the system can use thesummarization tables to obtain partial results for the events that arecovered by summarization tables, but may also have to search throughother events that are not covered by the summarization tables to produceadditional results. These additional results can then be combined withthe partial results to produce a final set of results for the query.This summarization table and associated techniques are described in moredetail in U.S. Pat. No. 8,682,925, issued on Mar. 25, 2014.

1.7.4 Accelerating Report Generation

In some embodiments, a data server system such as the SPLUNK® ENTERPRISEsystem can accelerate the process of periodically generating updatedreports based on query results. To accelerate this process, asummarization engine automatically examines the query to determinewhether generation of updated reports can be accelerated by creatingintermediate summaries. (This is possible if results from preceding timeperiods can be computed separately and combined to generate an updatedreport. In some cases, it is not possible to combine such incrementalresults, for example where a value in the report depends onrelationships between events from different time periods.) If reportscan be accelerated, the summarization engine periodically generates asummary covering data obtained during a latest non-overlapping timeperiod. For example, where the query seeks events meeting a specifiedcriteria, a summary for the time period includes only events within thetime period that meet the specified criteria. Similarly, if the queryseeks statistics calculated from the events, such as the number ofevents that match the specified criteria, then the summary for the timeperiod includes the number of events in the period that match thespecified criteria.

In parallel with the creation of the summaries, the summarization engineschedules the periodic updating of the report associated with the query.During each scheduled report update, the query engine determines whetherintermediate summaries have been generated covering portions of the timeperiod covered by the report update. If so, then the report is generatedbased on the information contained in the summaries. Also, if additionalevent data has been received and has not yet been summarized, and isrequired to generate the complete report, the query can be run on thisadditional event data. Then, the results returned by this query on theadditional event data, along with the partial results obtained from theintermediate summaries, can be combined to generate the updated report.This process is repeated each time the report is updated. Alternatively,if the system stores events in buckets covering specific time ranges,then the summaries can be generated on a bucket-by-bucket basis. Notethat producing intermediate summaries can save the work involved inre-running the query for previous time periods, so only the newer eventdata needs to be processed while generating an updated report. Thesereport acceleration techniques are described in more detail in U.S. Pat.No. 8,589,403, issued on Nov. 19, 2013, and U.S. Pat. No. 8,412,696,issued on Apr. 2, 2011.

System for Processing and Displaying Geographic Data

In addition to processing data related to computer performance andnetworking traffic, the above-described event-based framework can alsobe used to process other types of data, such as geographic data. Forexample, FIG. 7 illustrates a system for processing and displayinggeographic data in accordance with the disclosed embodiments. Thissystem includes a data store 103 containing geographic data, such aspopulation statistics associated with geographic regions. (Note thatdata store 103 illustrated in FIG. 7 is analogous to data store 103illustrated in FIG. 1.) It also includes a query processor 710configured to execute queries involving geographic data. (Note thatquery processor 710 illustrated in FIG. 7 is analogous to queryprocessor 404 illustrated in FIG. 4.) Results generated by queryprocessor 710 can be formatted using display mechanism 708 and thendisplayed to a user 702 through a display system 704. For example, asillustrated in FIG. 7, the results can be displayed using a choroplethmap 706, wherein each geographic region is shaded, patterned or coloredin proportion to a number of data points that fall into the geographicregion. The process of generating a choropleth map (or some otherrepresentation of geographic data) is described in more detail below.

Fast Point-in-Polygon Indexing

To generate a choropleth map, we first determine which data points fallinto each geographic region. This can be accomplished by using atechnique that solves the point-in-polygon (PIP) problem. The PIPproblem is typically solved using either the crossing number (CN)technique or the winding number (WN) technique. For example, the CNtechnique is depicted in FIG. 8 for a single polygon and single pointlocated at the end of the arrow labeled “5”. A ray is cast from theexterior of the polygon to the point being tested and the counter isincremented for each edge that is crossed. Although in FIG. 8 thecounter is shown as being incremented in the order in which the rayintersects the polygon's boundary segments, there is no requirement thatthe segments be tested in any particular order. Typically, the linesegments are iterated either clockwise or anticlockwise from anarbitrary “starting point” in the polygon boundary. The CN techniquedeclares a point to be inside the polygon if the total crossing count isodd. Intuitively this makes sense, because crossing into a boundary fromthe exterior places the ray inside the shape, and the ray can only exitthe shape by crossing the boundary again, thus flipping the count fromodd to even. Note that the ray may be cast in any direction; the choiceof a horizontal ray in FIG. 8 is arbitrary. Furthermore, it does notmatter whether the ray is cast from the point to infinity in anydirection, or from infinity to the point. The crossing count will be thesame regardless of whether the ray is cast inward or outward.

Because this disclosure focuses on accelerating the CN technique, andbecause the CN technique has the same computational complexity as the WNtechnique, we will not describe the WN technique further other than tosay that it iterates the vertices of the polygon and counts the numberof full turns around the point being tested.

There are a number of existing approaches to accelerating PIP testing.Perhaps the most common is to insert the polygons into a spatial datastructure such as an R-Tree. One example of this is the PostGIS™database. The R-Tree can be queried to retrieve a set of candidatepolygons based on rectangular bounding boxes surrounding each polygon.This eliminates polygons whose bounding boxes do not surround the pointin question. Each surviving candidate must then be tested using eitherCN or WN to determine if the point in question falls inside, outside oron the edge of the candidate. Because candidates may contain largenumbers of vertices, and because there may exist many candidates, the CNand WN techniques are still the bottleneck because they are both O(n)where n is the number of vertices in the polygon being tested. Morerecently, an approach has been proposed that builds an indexspecifically for PIP testing; however, this approach suffers from anexponential index build complexity of O(n^(2.6)). Both the R-Tree andthe above-described index-building approach rely on 10 in-memory treestructures. The performance of these in-memory tree structures whenstored on spinning disk media as opposed to RAM, depends on the depth ofthe tree because each link traversal in a tree search requires a seekoperation, and seek operations are slow on traditional spinning media.

The new indexing technique described in this disclosure involves 15casting horizontal rays from polygon segments onto a reference line andthen storing the set of segments intersected by the rays. For a givenpoint to be PIP tested, the set of intersected segments across allpolygons in the index can then be retrieved from the index, and feddirectly to the CN technique. This obviates the need to iterate theboundaries of the polygons to determine which segments intersect a rayemanating from the point to be tested.

FIG. 9 shows a polygon labeled “A”, and the projection of line segmentsS1 through S4 onto the Y-axis. Note that segment projections overlap onthe Y-axis, and the set of overlapping segments is constant over anycontinuous range of Y-values bounded by the projection of 2 successivepoints on the Y-axis. For instance, referring to FIG. 9, for thecontinuous range of Y-values [y2, y3) one can see that a horizontal rayfixed at any Y-value in the range will intersect the set of segments[S2, S4]. Therefore, an indexing mechanism can be designed such that fora given point to be PIP tested, the Y-coordinate of the point is used tolook up a corresponding set of intersected segments based upon which ofa finite set of Y-ranges the point's Y-value falls inside.

We now describe how to build such an index that comprises sortedY-ranges, and their corresponding set of intersected segments. Considera point data structure for describing the points as {segmentId, x, y,order}. The “order” field can assume the values 0 or C for “opening” or“closing”. An “opening” point for a segment corresponds to the pointwith the smaller Y-coordinate. A “closing” point is the segment's pointwith the larger Y-coordinate. For example, 10 the coordinate of S1 withthe smaller Y-value would be {S1, --, y1, 0} (the X-values are shown as“--”). By sorting the point structures based on their Y-value fromsmallest to largest, we obtain the ordering with ascending Y-valuesillustrated in Table 1.

TABLE 1 {S1, --, y1, O} {S4, --, y1, O} {S1, --, y2, C} {S2, --, y2, O}{S3, --, y3, O} {S4, --, y3, C} {S2, --, y4, C} {S3, --, y4, C}

By traversing this sorted set of point data structures from beginning toend, we can create a corresponding list of open segments (that is tosay, segments that would intersect a horizontal ray projected from pointx,y). As we traverse the listed point structures shown in Table 1, wecan perform the following operation: if the order field is “0”, then weadd the corresponding segment ID to the list of open segments. If theorder field is “C”, then we remove the corresponding segment ID from thelist. Hereafter, we call this the “opening and closing technique.”Following this simple procedure, the list of open segments is shown nextto each point data structure as is illustrated in Table 2 below.

For the list shown in Table 2, the final list for a given Y-valueprovides the set of open segments between the current Y-value and thenext Y value. Table 3 illustrates this by striking out the non-finalsegments.

TABLE 2 {S1, --, y1, O} : [S1] {S4, --, y1, O} : [S1, S4] {S1, --, y2,C} : [S4] {S2, --, y2, O} : [S2, S4] {S3, --, y3, O} : [S2, S3, S4] {S4,--, y3, C} : [S2, S3] {S2, --, y4, C} : [S3] {S3, --, y4, C} : [ ]

From the remaining adjacent Y-values (those not struck out), we canextract the Y-coordinates, and the open lists to create continuousY-ranges, and corresponding lists of open segments, as shown in Table 4,including the ranges from negativity infinity, and to positive infinityon the Y-axis.

TABLE 3

{S4, -- , y1,O}: [S1, S4]

{S2, -- , y2,O}: [S2, S4]

{S4, -- , y3,C}: [S2, S3]

{S3, -- , y4,C}: [ ]

TABLE 4 [ -INF, y1): [ ] [y1, y2): [S1, S4] [y2, y3): [S2, S4] [y3, y4):[S2, S3] [y4, INF): [ ]

FIG. 10 illustrates each of the 5 ranges, and a horizontal ray castthrough each range. One can easily validate from the image in FIG. 10that the list of open segments for each range, as shown in Table 4,accurately describes the set of segments intersected by a given ray. Forexample, the ray in range [y3, y4) intersects S2 and S3. For each rangeof Y-values, we have computed the set of intersections of a horizontalray. Therefore, the list of open segments is actually a set of rayintersections for any ray in the given range.

TABLE 5

{S2A, --, y1, O}: [S1A, S2A]

{S1A, --, y2, C}: [S2A, S3A]

{S3B, --, y3, O}: [S2A, S3A, S1B, S3B]

{S1B, --, y4, C}: [S2A, S3A, S3B, S2B]

{S3B, --, y5, C}: [S2A, S3A]

{S3A, --, y6, C}: [ ]

We can easily extend the above-described methodology to multiplepolygons by including the identity of the polygon in the segmentIdfield. For example, FIG. 11 illustrates the case in which multiplepolygons are projected onto the Y-axis. Polygon A has segments withidentity S1A, S2A, and S3A, and Polygon B has segments S1B, S2B, andS3B. Applying the above-described technique to the polygons shown inFIG. 10 yields the result that is displayed in Table 5. A comparison ofTable 3 and Table 5 shows that, as one would expect, as more polygonsare present, the set of ray intersections becomes larger for a givenrange.

We now describe a technique for minimizing the storage of rayintersection lists because the lists themselves can grow large and tendto be duplicative with respect to their neighboring lists. This isaccomplished by storing 1 of every K ray intersection lists, and usingthe point data structure to update a stored intersection list atruntime. That is, the intersection list is stored modulo K. In thisvariation, the point data structure must be stored for each point and isused at runtime to update a stored ray intersection set to the desiredYrange. For example, Table 6 below illustrates polygons of FIG. 11stored with K=3 (modulo 3). Note how only one of every threeintersection sets is not struck out, and also note that no pointstructures are struck out. The modulo parameter K provides a practicaltradeoff between space in the index and the runtime cost of updating astored ray to the desired Y-range. In practice, a K value as large as1024 can be used. For very large K values the storage space required forthe index is approximately the same as for the polygons themselves.

We now describe the process of determining the set of intersectedsegments of a horizontal ray at an arbitrary height, given the modulostyle of index storage. Given a desired ray height of Y, the sortedpoint structure is binary searched to determine the point having astored ray, and a Y-value less than or equal to the desired Y-value. Inessence, the system retrieves a ray that is parallel to, but lower thanthe desired ray. This lower ray is then raised to the desired heightusing the opening and closing technique. When a point is reached havinga Y-value greater than the desired height, the technique terminates.

TABLE 6 {S1A, --, y1, O}: [S1A] {S2A, --, y1, O}:  

{S3A, --, y2, O}:  

{S1A, --, y2, C}: [S2A, S3A] {S1B, --, y3, O}:  

{S3B, --, y3, O}:  

{S2B, --, y4, O}: [S2A, S3A, S1B, S3B, S2B] {S1B, --, y4, C}:  

{S2B, --, y5, C}:  

{S3B, --, y5, C}: [S2A, S3A] {S2A, --, y6, C}:  

{S3A, --, y6, C}:  

For example, consider a point at location {x,y} for which we want 5 toanswer the PIP question. Assume Y falls in the range [y4, y5) of Table6. Binary searching the point structure yields {S2B, --, y4, O}: [S2A,S3A, S1B, S3B, S2B] because the point structure and ray are lower thanor equal to the desired y. Correcting the ray using the following pointstructure {S1B, --, y4, C} updates the ray to [S2A, S3A, S3B, S2B],which is the correct set of intersections 10 for a ray falling in therange [y4, y5). The next point structure, {S2B, --, y5, C} causes theupdating to stop, because y5 is greater than the desired height.

The preceding disclosure describes how to retrieve a pre-cast ray fromthe index. Also, for the case of modulo indexing, we showed how to“correct” a stored ray to raise its height to the desired range. We nowdescribe the step of how to apply the CN technique to the ray. The raysretrieved (and possibly corrected) from the index are cast from negativeinfinity in the X direction toward positive infinity in the X direction.The intersection sets, such as [S2A, S3A, SIB, S3B, S2B] are made ofsegment IDs that include the polygon identity (for instance “A” and“B”). Therefore, before feeding segments to the CN technique, we mustfirst group them by polygonid. This is illustrated in FIG. 12. While wehave heretofore described the elements of the intersection list assimple lists of segmentId values, we in fact store both the opening andclosing point of each segment in conjunction with the segmentId. Thereason for this is that a ray cast from negative infinity to positiveinfinity effectively includes two ray tests (we can think of this as aray originating at the point to test, and emanating to positive infinityand the second ray originating at the point to test emanating tonegative infinity). Because the CN test casts a single ray, we need thetwo points defining each line segment to determine which segments do notintersect the test ray, which we arbitrarily choose to be the oneemanating to positive infinity.

Using the data illustrated in FIG. 12, the CN test first groups thesegments by polygon. Then, for each set, the CN test counts segmentsthat are intersected by the ray emanating to positive infinity. Fromthis point, the output of the CN technique is of the standard form: ifthe count is odd, the point in question was inside the given polygon. Ifthe count is even, the point in question was outside the given polygon.

In the technique presented so far, a ray must be retrieved (andcorrected as needed) for each point to be tested. In practice,retrieving the ray and correcting it takes substantially longer (byorders of magnitude) than performing the CN technique on the retrievedset. The fact that the intersection set is invariant for a given range[ymin, ymax) allows for amortization of the intersection set retrievaland correction costs when a batch of points needs to be PIP tested. Thiscan be accomplished by first sorting the points to be batch tested basedon their Y-value. While processing the first point in the now-sortedbatch, the corresponding ray can be retrieved and corrected from theindex. For the second point, and all subsequent points to be tested, theray can be incrementally corrected. That is to say, if the subsequentpoint to be tested still falls within the [ymin, ymax) range of thecorrected ray, then no further action is required, and the CN technique,is again applied directly to the subsequent point. Subsequent points areprocessed until a point is encountered whose Y-value surpasses the upperextent of the ray's valid range (ymax). At this point, rather thanperforming a brand new query for a ray (which would require a binarysearch), one can simply continue to apply correction to the ray whichrequires forward iteration in the index, but not binary searching.

Note that in addition to determining which points fall within eachpolygon, the above-described technique can be modified to determinewhether a given shape overlaps, intersects or encloses another shape byperforming the technique for the points that define the given shape.

Displaying Geographic Data

FIG. 13 presents a flow chart illustrating the process of displayinggeographic data in accordance with the disclosed embodiments. At thestart of the process, the system obtains a set of polygons that define aset of geographic regions, wherein each polygon comprises line segmentsthat define a border of the polygon, and wherein each line segment isdefined by coordinates for two endpoints of the line segment (step1302). Next, the system projects rays from the endpoints of the linesegments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line (step1304). (In some embodiments, the reference line is the Y-axis in aCartesian coordinate system and the rays are projected horizontallyparallel to the X-axis.) Then, for each interval between pairs ofconsecutive intersection points on the reference line, the system keepstrack of open line segments that project onto the interval (step 1306).

Next, for each data point in a set of data points to be processed, thesystem identifies a relevant interval on the reference line that thedata point projects onto (step 1308), and performs a crossing number(CN) operation by counting intersections between a ray projected fromthe data point and open line segments associated with the relevantinterval to identify zero or more polygons that the data point fallsinto (step 1310). The intersections can be detected by performing anintersection test using an equation for the line segment (y=mx+b) to seewhether a ray projected from positive infinity to the data pointintersects the line segment. If the reference line is the Y-axis, thesystem can perform a simple initial filtering test to see whether bothX-coordinates of the line segment are less than the data point'sX-coordinate. This enables the system to throw out line segments thatare obviously not intersected by the ray. The system then increments acount for each polygon that the data point falls into (step 1312).

Finally, the system displays the set of geographic regions, wherein eachpolygon that defines a geographic region is marked to indicate a numberof data points that fall into the polygon (step 1314).

Performing a Geofencing Operation

FIG. 14 presents a flow chart illustrating the process of performing ageofencing operation in accordance with the disclosed embodiments. Atthe start of this process, the system obtains a set of polygons thatdefine a set of geographic regions, wherein each polygon comprises linesegments that define a border of the polygon, and wherein each linesegment is defined by coordinates for two endpoints of the line segment(step 1402). Next, the system projects rays from the endpoints of theline segments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line (step1404). Then, for each interval between pairs of consecutive intersectionpoints on the reference line, the system keeps track of open linesegments that project onto the interval (step 1406).

Next, for a data point to be processed, the system identifies a relevantinterval on the reference line that the data point projects onto (step1408). The system subsequently performs a crossing number (CN) operationby counting intersections between a ray projected from the data pointand open line segments associated with the relevant interval to identifyzero or more polygons that the data point falls into (step 1410).Finally, the system performs a geofencing operation based on theidentified zero or more polygons that the data point falls into (step1412). For example, the geofencing operation can involve sending anotification to a user when the user's location (obtained from theuser's phone) indicates that the user has crossed a geofence boundaryassociated with a restricted security area.

Processing a Query Involving Geographic Information

FIG. 15 presents a flow chart illustrating the processing of a queryinvolving geographic information in accordance with the disclosedembodiments. First, the system receives the query to be processed,wherein the query is associated with a set of geographic regions (step1502). Next, the system uses a late-binding schema generated from thequery to retrieve data points from a set of events containing previouslygathered data (step 1504). For example, if the query asks to count thepopulation in each state in the United States, the system can use alate-binding schema to retrieve residence locations for people in theUnited States from a set of event data. As is described in more abovewith reference to FIG. 4, a late-binding schema specifies “extractionrules” that are used to extract values from specific fields in eventsduring query processing. In contrast to a conventional schema for adatabase system, a late-binding schema is not defined at data ingestiontime. Instead, the late-binding schema can be developed on an ongoingbasis until the time the query is actually executed.

Next, for each for each data point in a set of data points, the systemidentifies zero or more geographic regions in the set of geographicregions that the data point falls into (step 1506). Finally, the systemdisplays the set of geographic regions, wherein each polygon thatdefines a geographic region is marked to indicate a number of datapoints that fall into the polygon (step 1508).

Efficiently Solving the 3D PIP Problem

We now describe a solution to the 3D PIP problem, which is ageneralization of the 2D PIP problem, that can similarly be applied to adata set stored in an event-processing framework. To understand the 3DPIP problem, consider the triangular prism shown in FIG. 16. The blackdot in FIG. 16 represents the point of a ray projecting directly at theeye of the viewer. The ray is parallel to the Z-axis and orthogonal tothe X- and Y-axes. The five faces of the triangular prism are labeled 1,2, 3, 4 and 5. Note that sides 3, 4, and 5 are not directly visible fromthe viewpoint. Consider the projection of the 5 faces onto the X-Y planeby setting the Z-coordinate of each vertex of the prism to zero.

This results in five flat shapes on the X-Y plane. We can now considerthe dot to represent the intersection of the ray originating at negativeinfinity in the Z direction, with the X-Y plane. Using all thetechniques presented earlier for the case of two dimensions, we candetermine which of the five polygons encloses the dot (the dot is apoint). The polygons that enclose the dot correspond to faces of theprism through with the ray passed.

The CN algorithm can then be applied once again, but this time to thefaces of the prism that the ray intersects, which we have just shown howto determine by projecting the faces onto two dimensions. As was thecase with multiple polygons, the technique works with multipleintersecting 3D volumes (also referred to as “polyhedral”). Each segmentof a face of a volume has an associated ID that encodes the volume ID,the face/polygon ID, and a globally unique segment ID. Faces thatintersect the ray are first grouped by their volume ID; then, thecrossings are counted on a per-volume basis.

More specifically, FIG. 17 presents a flow chart illustrating theprocess of solving a three-dimensional PIP problem in accordance withthe disclosed embodiments. During operation, the system obtains a set ofpolyhedra, wherein each polyhedron in the set comprises polygons thatdefine faces of the polyhedron, wherein each polygon comprises linesegments that define a border of the polygon, and wherein each linesegment is defined by three-dimensional coordinates for two endpoints ofthe line segment (step 1702). Next, the system projects polygons in theset of polyhedra onto projected polygons in a reference plane (step1704). If the reference plane is the plane containing the X- and Y axes,this projection simply involves setting the Z-coordinates of endpointsof the line segments that comprise the set of polyhedra to zero.

Then, for each data point in a set of one or more data points to beprocessed, the system does the following. First, the system projects thedata point onto a projected data point in the reference plane (step1706). If the reference plane is the plane containing the X- and Y-axes,this projection simply involves setting the Z-coordinate of the datapoint to zero. The system then performs a two-dimensionalpoint-in-polygon (PIP) operation in the reference plane to determinewhich projected polygons the projected data point falls into (step1708). This can involve using the two-dimensional PIP techniquedisclosed above. Next, for each projected polygon that the projecteddata point falls into, the system performs a three-dimensional crossingnumber (CN) operation by counting intersections between a ray projectedfrom the corresponding data point in a direction orthogonal to thereference plane and polyhedral faces corresponding to projectedpolygons, to identify zero or more polyhedra that the data point fallsinto (step 1710).

To determine whether a ray projected from the data point intersects apolyhedral face, we first determine the equation of the plane thatcontains the polyhedral face. This can be accomplished by obtainingthree noncollinear points in the plane, for example by obtaining threenon-collinear endpoints from the line segments that comprise thepolyhedral face. We then can then use these three non-collinear pointsto determine the equation of the plane that contains the polyhedral faceas follows. Assume the three points are p, q, r, wherein p=(p_(x),p_(y), P_(z)), q=(q_(x), q_(y), q_(z)), r=(r_(x), r_(y), r_(z)). Wefirst determine two vectors in the plane based on the three points bydetermining the vector from p to q=<(q_(x)−p_(x)), (q_(y)−p_(y)),(q_(z)−p_(z))> and determining the vector from q to r=<(r_(x)−q_(x)),(r_(y)−q_(y)), (r_(z)−q_(z))>. Next, we take the cross-product of thesetwo vectors to find the normal vector to the plane that contains thepolyhedral face <a,b,c>. The equation of the plane that contains thepolyhedral face is then given by a(x−x₀)+b(y−y₀)+c(z−z₀)=0, wherein (x₀,y₀, z₀) is a point in the plane and the vector <a,b,c> is a normalvector to the plane. We then plug a, b and c and the coordinates of oneof the points p, q or r into this equation to obtain the equation forthe plane that contains the polyhedral face.

We can then determine whether a ray projected upwards (in the positive Zdirection) from the data point intersects the polyhedral face by firstplugging the X- and Y-values of the data point into the equation for theplane that contains the polyhedral face and then solving for theZ-value. Note that this computed Z value is the Z value for theintersection point between the polyhedral face and a line that isparallel to the Z axis and extends both upwards and downwards from thedata point. (Also, note that the solution to the 2D PIP problem ensuresthat the X and Y values for the data point will fall within the X and Yboundaries of the polyhedral face.) If the computed Z-value forintersection point is greater than the Z-value for the data point, a rayprojected upwards from the data point intersects the polyhedral face. Onthe other hand, if the computed Z value for the intersection point islower than the Z value for the data point, the ray projected upwardsfrom the data point will not intersect the polyhedral face because thepolyhedral face lies below the data point. Hence, this techniqueeffectively discards polyhedral faces that lie below the data point.

Finally, the system generates a visual representation of the set ofpolyhedra, wherein the visual representation of each polyhedron isaffected by data points that fall into the polyhedron (step 1712), andpresents the visual representation to a user (step 1714).

Applications of Solutions to the 3D PIP Problem

The above-described solution to the 3D PIP problem can be applied in anumber of important use-cases. As mentioned above, a solution to the 3DPIP problem can be applied while displaying a 3D data set comprising aset of polyhedra, wherein the wherein the visual representation of eachpolyhedron is affected by data points that fall into the polyhedron.(This is effectively a generalization of the 2D Choropleth problem tothree dimensions.)

Other applications involve considering a moving point in a 3D space, anddetermining which of a set of fixed volumes (polyhedra) the point may beinside. For instance, in an aircraft navigation system, the moving pointcould be the aircraft and the fixed polyhedra could be pre-definedair-corridors. In this use-case, we want to ensure the aircraft stayswithin its assigned air corridor. If the aircraft deviates from itsassigned air corridor and enters other air corridors, we need to knowwhich ones, so we can alert other pilots and ground control.

A related application is in collision detection mechanisms for videogames. In a video game, a vehicle might travel through 3D Cartesianspace, and the fixed objects (or at least relatively fixed) couldinclude the surface of a planet, or buildings attached to the surface.In a first-person shooter video game, the fixed objects could includerooms, corridors or various boundaries. To the extent that anything inthe video game can be approximated as a point (e.g., a bullet)point-in-volume testing can be applied.

In some case, moving objects can be considered to be fixed objects bysurrounding them with a bounding cube, and performing point-in volumetesting on the object only when the point enters the bounding cube.

Also note that the PIP index could be built relative to a referenceframe that could itself be rotated. This would allow for a complex butfixed-shape object like a spacecraft to be indexed, yet still allowed tomove and rotate. In other words, the location of the point could betranslated to the indexing reference frame of the object, which allowsPIP indexing to take place on a per-complex object basis, therebyallowing a point to be tested against a complex moving object.

The above-described techniques can also be used with other 3D data sets.For example, the 3D data sets can comprise weather models at multiplealtitudes that enable the user to visualize how weather systems evolvein three dimensions, and a solution to the 3D PIP problem can be used todetermine where an aircraft is located with respect to nearby weathersystems. The 3D data sets can also include a detailed cell phonecoverage map by floor within a building that enables the user toidentify dead zones for cell phone coverage within the building. In thisexample, a solution to the 3D PIP problem can be used to determinewhether a cell phone user is located in a specific dead zone. The 3Ddata sets can additionally comprise sound models for different regionsof a room, and a solution to the 3D PIP problem can be used to determinewhere a listener is located with respect to these different regions.

The preceding description was presented to enable any person skilled inthe art to make and use the disclosed embodiments, and is provided inthe context of a particular application and its requirements. Variousmodifications to the disclosed embodiments will be readily apparent tothose skilled in the art, and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the disclosed embodiments. Thus, the disclosedembodiments are not limited to the embodiments shown, but are to beaccorded the widest scope consistent with the principles and featuresdisclosed herein. Accordingly, many modifications and variations will beapparent to practitioners skilled in the art. Additionally, the abovedisclosure is not intended to limit the present description. The scopeof the present description is defined by the appended claims.

The data structures and code described in this detailed description aretypically stored on a computer-readable storage medium, which may be anydevice or medium that can store code and/or data for use by a system.The computer-readable storage medium includes, but is not limited to,volatile memory, non-volatile memory, magnetic and optical storagedevices such as disk drives, magnetic tape, CDs (compact discs), DVDs(digital versatile discs or digital video discs), or other media capableof storing code and/or data now known or later developed.

The methods and processes described in the detailed description sectioncan be embodied as code and/or data, which can be stored on anontransitory computer-readable storage medium as described above. Whena system reads and executes the code and/or data stored on thenon-transitory computer readable storage medium, the system performs themethods and processes embodied as data structures and code and storedwithin the non-transitory computer-readable storage medium.

Furthermore, the methods and processes described above can be includedin hardware modules. For example, the hardware modules can include, butare not limited to, application-specific integrated circuit (ASIC)chips, field programmable gate arrays (FPGAs), and otherprogrammable-logic devices now known or later developed. When thehardware modules are activated, the hardware modules perform the methodsand processes included within the hardware modules.

Having thus described the invention, what is claimed is:
 1. Acomputer-implemented method for generating a visual representation of aset of polyhedra in a three-dimensional space, comprising: projectingpolygons that define faces of each polyhedron in the set of polyhedraonto projected polygons in a reference plane; for each data point in aset of one or more data points in three-dimensional space to beprocessed to determine what polyhedra the data point falls into,projecting the data point in three-dimensional space onto a projecteddata point in the reference plane, performing a two-dimensionalpoint-in-polygon (PIP) operation in the reference plane using an indexof open line segments to determine which projected polygons theprojected data point falls into, and for each projected polygon that theprojected data point falls into, performing a three-dimensional crossingnumber (CN) operation to identify zero or more polyhedra that the datapoint falls into; generating a visual representation of the set ofpolyhedra, wherein the visual representation of each polyhedron isaffected by data points that fall into the polyhedron; and presentingthe visual representation to a user.
 2. The computer-implemented methodof claim 1, wherein each polygon, which defines a face of a polyhedronin the set of polyhedra, comprises line segments that define a border ofthe polygon, and wherein each line segment is defined bythree-dimensional coordinates for endpoints of the line segment; andwherein performing the two-dimensional point-in-polygon (PIP) operationfor each projected data point in the reference plane comprises:projecting rays in the reference plane from endpoints of projected linesegments of the projected polygons onto a reference line in thereference plane, wherein the rays are projected in a directionorthogonal to the reference line to form intersection points with thereference line, and storing the intersection points in the index; foreach interval between pairs of consecutive intersection points on thereference line, storing in the index open line segments that projectonto the interval; identifying a relevant interval on the reference linethat the projected data point projects onto; and performing atwo-dimensional crossing number (CN) operation by counting intersectionsbetween a projected ray from the projected data point and the open linesegments, the number of intersections being determined using the openline segments stored in the index and associated with the relevantinterval to identify zero or more projected polygons that the projecteddata point falls into in the reference plane.
 3. Thecomputer-implemented method of claim 1, wherein performing thethree-dimensional CN operation includes counting intersections between aray projected from the corresponding data point in a directionorthogonal to the reference plane and polyhedral faces corresponding toprojected polygons, to identify the zero or more polyhedra that the datapoint falls into.
 4. The computer-implemented method of claim 1, whereinthe reference plane is a plane that includes an X- and a Y-coordinateaxis.
 5. The computer-implemented method of claim 1, wherein thereference line is an X-coordinate axis.
 6. The computer-implementedmethod of claim 1, wherein the reference line is a Y-coordinate axis. 7.The computer-implemented method of claim 1, wherein projecting the datapoint onto the projected data point in the reference plane includessetting a Z-coordinate for the data point to zero.
 8. Thecomputer-implemented method of claim 1, wherein projecting polygons inthe set of polyhedra onto projected polygons in the reference planeincludes setting Z-coordinates for line segments of the set of polyhedrato zero.
 9. The computer-implemented method of claim 1, whereingenerating the visual representation for the set of polyhedra includesone of: generating a visual representation for a computer game thatincludes the set of polyhedra; and generating a visualization for amedical-imaging system that includes the set of polyhedra.
 10. Thecomputer-implemented method of claim 1, wherein prior to processing theset of data points, the method further comprises obtaining the set ofdata points by: receiving a query to be processed, wherein the query isassociated with the set of polyhedral; and using a late-binding schemagenerated from the query to retrieve the set of data points from a setof events containing previously gathered data.
 11. Thecomputer-implemented method of claim 1, wherein the method operates inan aircraft navigation system; wherein each data point represents anaircraft; wherein each polyhedron represents an air corridor or aportion of an air corridor; and wherein the method additionallydetermines when each aircraft enters or leaves an air corridor.
 12. Thecomputer-implemented method of claim 1, wherein the method operates in avideo game; wherein each data point represents a vehicle that can bemanned or unmanned; wherein each polyhedron represents a physical objectin the video game; and wherein the method additionally determines wheneach vehicle collides with or is in danger of colliding with a physicalobject.
 13. A non-transitory computer-readable storage medium storinginstructions that when executed by a computer cause the computer toperform a method for generating a visual representation of a set ofpolyhedra in a three-dimensional space, the storage medium including:instructions for projecting polygons that define faces of eachpolyhedron in the set of polyhedra onto projected polygons in areference plane; for each data point in a set of one or more data pointsin three-dimensional space to be processed to determine what polyhedrathe data point falls into, instructions for projecting the data point inthree-dimensional space onto a projected data point in the referenceplane, performing a two-dimensional point-in-polygon (PIP) operation inthe reference plane using an index of open line segments to determinewhich projected polygons the projected data point falls into, and foreach projected polygon that the projected data point falls into,instructions for performing a three-dimensional crossing number (CN)operation to identify zero or more polyhedra that the data point fallsinto; instructions for generating a visual representation of the set ofpolyhedra, wherein the visual representation of each polyhedron isaffected by data points that fall into the polyhedron; and instructionsfor presenting the visual representation to a user.
 14. Thenon-transitory computer-readable storage medium of claim 13, whereineach polygon, which defines a face of a polyhedron in the set ofpolyhedra, comprises line segments that define a border of the polygon,and wherein each line segment is defined by three-dimensionalcoordinates for endpoints of the line segment; and wherein theinstructions for performing the two-dimensional point-in-polygon (PIP)operation for the projected data point in the reference plane comprise:instructions for projecting rays in the reference plane from endpointsof projected line segments of the projected polygons onto a referenceline in the reference plane, wherein the rays are projected in adirection orthogonal to the reference line to form intersection pointswith the reference line, and storing the intersection points in theindex; for each interval between pairs of consecutive intersectionpoints on the reference line, instructions for storing in the index openline segments that project onto the interval; instructions foridentifying a relevant interval on the reference line that the projecteddata point projects onto; and instructions for performing atwo-dimensional crossing number (CN) operation by counting intersectionsbetween a projected ray from the projected data point and the open linesegments, the number of intersections being determined using the openline segments stored in the index and associated with the relevantinterval to identify zero or more projected polygons that the projecteddata point falls into in the reference plane.
 15. The non-transitorycomputer-readable storage medium of claim 13, wherein the instructionsfor performing the three-dimensional CN operation includes instructionsfor counting intersections between a ray projected from thecorresponding data point in a direction orthogonal to the referenceplane and polyhedral faces corresponding to projected polygons, toidentify the zero or more polyhedra that the data point falls into. 16.The non-transitory computer-readable storage medium of claim 13, whereinthe reference plane is a plane that includes an X- and a Y-coordinateaxis.
 17. The non-transitory computer-readable storage medium of claim13, wherein the reference line is an X-coordinate axis.
 18. Thenon-transitory computer-readable storage medium of claim 13, wherein thereference line is a Y-coordinate axis.
 19. The non-transitorycomputer-readable storage medium of claim 13, wherein the instructionsfor projecting the data point onto the projected data point in thereference plane include instructions for setting a Z-coordinate for thedata point to zero.
 20. The non-transitory computer-readable storagemedium of claim 13, wherein the instructions for projecting polygons inthe set of polyhedra onto projected polygons in the reference planeinclude instructions for setting Z-coordinates for line segments of theset of polyhedra to zero.
 21. The non-transitory computer-readablestorage medium of claim 13, wherein the instructions for generating thevisual representation for the set of polyhedra include one of:instructions for generating a visual representation for a computer gamethat includes the set of polyhedra; and instructions for generating avisualization for a medical-imaging system that includes the set ofpolyhedra.
 22. The non-transitory computer-readable storage medium ofclaim 13, wherein the storage medium additionally includes instructionsfor obtaining the set of data points prior to processing the set of datapoints, wherein the instructions for obtaining the set of data pointsincludes: instructions for receiving a query to be processed, whereinthe query is associated with the set of polyhedra; and instructions forusing a late-binding schema generated from the query to retrieve the setof data points from a set of events containing previously gathered data.23. The non-transitory computer-readable storage medium of claim 13,wherein the instructions operate in an aircraft navigation system;wherein each data point represents an aircraft; wherein each polyhedronrepresents an air corridor or a portion of an air corridor; and whereinthe method additionally determines when each aircraft enters or leavesan air corridor.
 24. The non-transitory computer-readable storage mediumof claim 13, wherein the instructions operate in a video game; whereineach data point represents a vehicle that can be manned or unmanned;wherein each polyhedron represents a physical object in the video game;and wherein the method additionally determines when each vehiclecollides with or is in danger of colliding with a physical object.
 25. Asystem that generates a visual representation of a set of polyhedra in athree-dimensional space, comprising: at least one processor and at leastone associated memory; and a display mechanism that executes on the atleast one processor, wherein during operation, the display mechanism:projects polygons that define faces of each polyhedron in the set ofpolyhedra onto projected polygons in a reference plane; for each datapoint in a set of one or more data points in three-dimensional space tobe processed to determine what polyhedra the data point falls into,projects the data point in three-dimensional-space onto a projected datapoint in the reference plane, performs a two-dimensionalpoint-in-polygon (PIP) operation in the reference plane using an indexof open line segments to determine which projected polygons theprojected data point falls into, and for each projected polygon that theprojected data point falls into, performs a three-dimensional crossingnumber (CN) operation to identify zero or more polyhedra that the datapoint falls into; generates a visual representation of the set ofpolyhedra, wherein the visual representation of each polyhedron isaffected by data points that fall into the polyhedron; and presents thevisual representation to a user.
 26. The system of claim 25, whereineach polygon, which defines a face of a polyhedron in the set ofpolyhedra, comprises line segments that define a border of the polygon,and wherein each line segment is defined by three-dimensionalcoordinates for endpoints of the line segment; and wherein whileperforming the two-dimensional point-in-polygon (PIP) problem for theprojected data point in the reference plane, the system: projects raysin the reference plane from endpoints of projected line segments of theprojected polygons onto a reference line in the reference plane, whereinthe rays are projected in a direction orthogonal to the reference lineto form intersection points with the reference line, and stores theintersection points in the index; for each interval between pairs ofconsecutive intersection points on the reference line, stores in theindex open line segments that project onto the interval; identifies arelevant interval on the reference line that the projected data pointprojects onto; and performs a two-dimensional crossing number (CN)operation by counting intersections between a projected ray from theprojected data point and the open line segments, the number ofintersections being determined using the open line segments stored inthe index and associated with the relevant interval to Identify zero ormore projected polygons that the projected data point falls into in thereference plane.
 27. The system of claim 25, wherein while performingthe three-dimensional CN operation, the system counts intersectionsbetween a ray projected from the corresponding data point in a directionorthogonal to the reference plane and polyhedral faces corresponding toprojected polygons, to identify the zero or more polyhedra that the datapoint falls into.
 28. The system of claim 25, wherein the referenceplane is a plane that includes an X- and a Y-coordinate axis.
 29. Thesystem of claim 25, wherein the reference line is an X-coordinate axis.30. The system of claim 25, wherein the reference line is a Y-coordinateaxis.